Skip to content

Fix CI (Python + MATLAB) and three small bugs in stats - #379

Merged
zihuaihuai merged 5 commits into
masterfrom
fix/ci-and-bugs
May 4, 2026
Merged

Fix CI (Python + MATLAB) and three small bugs in stats#379
zihuaihuai merged 5 commits into
masterfrom
fix/ci-and-bugs

Conversation

@zihuaihuai

@zihuaihuai zihuaihuai commented May 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Brings CI back to green (Python + MATLAB suites have been red on master) and folds in three high-severity correctness fixes from a recent audit. All 9 required checks now pass on this branch.

CI

Python tests (unittests matrix) — were aborting at test collection with ModuleNotFoundError: No module named 'pkg_resources'. Investigation produced a chain of issues:

  1. Default setuptools on Python 3.11/3.12 is now ≥81, which dropped pkg_resources. abagen 0.1.3 still imports pkg_resources.
  2. Pinning setuptools<81 exposed a numpy 2.x regression: float(size_1_array) raises TypeError: only 0-dimensional arrays can be converted to Python scalars.
  3. abagen 0.1.3 also calls DataFrame.groupby(sid, axis=1) which pandas 2.1 removed.
  4. Pinning pandas<2.1 resolves to pandas 2.0.x whose C extensions were built against numpy 1.x ABI → numpy.dtype size changed at import.
  5. pandas 2.0.x has no Python 3.12 wheels and cannot be built from source there.

Fixes:

  • Bumped actions/checkout@v4 and actions/setup-python@v5.
  • Pinned setuptools<81 (with re-install after the editable install in case it gets bumped back).
  • Pinned numpy<2,>=1.21 and pandas<2.1,>=1.3 in both setup.py and requirements.txt so the entire stack stays on numpy-1.x ABI until abagen drops pkg_resources/groupby(axis=).
  • Replaced float(np.sqrt(... / vf)) with .item() in _t_test.py:175 — works on numpy 1.x and 2.x.
  • Dropped Python 3.12 from the matrix (incompatible with the pandas<2.1 pin); added a comment so it can be re-added once abagen is patched.

MATLAB tests (matlab_unit_test matrix) — every test_precomputed/* errored with MATLAB:Python:PythonUnavailable because the workflow installed Python 3.8 but never told MATLAB about it via pyenv. Plus matlab-actions/setup-matlab@v0 is end-of-life.

  • Bumped to matlab-actions/setup-matlab@v2 and matlab-actions/run-command@v2; pyenv('Version', ...) is now called before runtests.
  • Switching to run-command lost the implicit addpath from run-tests, so added addpath(genpath('brainstat_matlab')) explicitly.
  • Pair MATLAB versions with compatible Python: R2022a ↔ 3.9, latest ↔ 3.10. Dropped the unsupported R019b matrix entry (setup-matlab v2 needs R2020a+).

Bugs

  • brainstat/stats/SLM.py — from cmath import sqrt was importing the complex square root. Switched to math.sqrt.
  • brainstat/stats/SLM.py — fetch_template_surface was being imported twice; kept the higher-level one.
  • brainstat/stats/_t_test.py — sys.exit(...) would kill the host process when contrast is non-estimable. Replaced with raise ValueError(...).

Test plan

  • python -m pytest brainstat/tests/test_t_test.py — 16/16 pass locally.
  • All 9 CI checks pass: Python 3.10/3.11 × ubuntu/macos/windows, MATLAB R2022a/latest.
  • Lift the numpy/pandas/3.12 pins once abagen drops pkg_resources and the groupby(axis=) call.

Follow-ups (not in this PR)

  • File an upstream issue against abagen tracking the pkg_resources and groupby(axis=) removals so the pins here can come off.
  • Consider re-enabling the commented-out mypy job after [REF] Remove #type: ignore from SLM class #199 unblocks.

@zihuaihuai
Fix CI and assorted bugs
fef045b 
CI:
- python_unittests: bump checkout to v4 and setup-python to v5; install
  setuptools/wheel explicitly before and after the editable install so
  abagen's `pkg_resources` import does not break test collection on
  Python 3.12 (where setuptools is no longer in the venv by default).
- MATLAB_unittests: bump checkout to v4, setup-python to v5,
  setup-matlab to v2 and switch to run-command v2 so we can call
  pyenv() before runtests; without it test_precomputed/load_pkl errors
  with MATLAB:Python:PythonUnavailable on the GitHub runners. Drop the
  unsupported R019b matrix entry (setup-matlab v2 requires R2020a+);
  test against R2022a (Python 3.9) and latest (Python 3.10) instead.

Bugs:
- SLM.py: import sqrt from math, not cmath (cmath returns complex
  values; the code uses real sqrt only).
- SLM.py: drop the duplicate fetch_template_surface import — it is
  re-exported from brainstat.datasets and was being imported twice.
- _t_test.py: replace sys.exit("Contrast is not estimable") with
  raise ValueError. Library code should never terminate the host
  process; remove the now-unused `import sys`.
@github-actions github-actions Bot added Github Related to the Github repository Python On the Python implementation labels May 4, 2026
zihuaihuai added 4 commits May 4, 2026 10:23
@zihuaihuai
CI: pin setuptools<81 and addpath brainstat_matlab in MATLAB tests
186ccfe 
Two follow-ups after first CI run on this branch:

- Python jobs were still failing with ModuleNotFoundError: pkg_resources.
  setuptools 82.0.1 was being installed, but setuptools 81+ no longer
  exposes the legacy pkg_resources module on import. abagen 0.1.3
  (resolved from `abagen>=0.1` in requirements.txt) still does
  `from pkg_resources import resource_filename` at import time, so test
  collection of brainstat/tests/test_genetics.py blows up. Pin
  setuptools<81 in CI; force-reinstall after the editable install in
  case it gets bumped back. Real fix is upstream — when abagen drops
  pkg_resources, this pin can go.
- MATLAB jobs were failing with `Undefined function fetch_template_surface`
  (and friends): switching from run-tests@v2 to run-command@v2 lost the
  implicit `addpath(source-folder)` that run-tests does. Add an
  explicit `addpath(genpath('brainstat_matlab'))` before runtests.
@zihuaihuai
CI: handle numpy 2.x scalar conversion and pin pandas <2.1
e092dd2 
After the previous CI fix, jobs on Python 3.11/3.12 surface two more
upstream-version regressions:

- _t_test.py:175 — `float(np.sqrt(... / vf))` raises TypeError on numpy
  >=2.0 when the inner expression is a size-1 ndarray (numpy 2 dropped
  the implicit 1d→scalar conversion). Switch to `.item()`, which
  returns a native Python scalar in both numpy 1.x and 2.x.
- abagen 0.1.3 calls `DataFrame.groupby(sid, axis=1)` in
  `abagen/probes_.py`, which raises in pandas 2.1+ where the `axis`
  keyword was removed. Until abagen ships a patched release, pin
  `pandas<2.1` in both setup.py install_requires and requirements.txt.
@zihuaihuai
CI: also pin numpy<2 alongside pandas<2.1
902cc1a 
pandas<2.1 resolves to 2.0.x, whose C extensions were built against the
numpy 1.x ABI. Installing it alongside numpy 2.x produces
`ValueError: numpy.dtype size changed, may indicate binary
incompatibility. Expected 96 from C header, got 88 from PyObject` at
import time, killing test collection across every matrix entry. Pin
numpy<2 so the entire stack stays on numpy 1.x ABI until abagen is
patched and the pandas pin can be lifted.
@zihuaihuai
CI: drop Python 3.12 from matrix while pandas is pinned <2.1
3741784 
pandas 2.0.x has no Python 3.12 wheels (3.12 support landed in pandas
2.1), so pip falls back to building from source — which fails because
the build env doesn't have pkg_resources. With pandas pinned <2.1
(needed for the abagen groupby workaround), 3.12 simply can't be
satisfied. Drop it from the matrix; add back once abagen is patched.
@zihuaihuai
zihuaihuai merged commit ec7e96c into master May 4, 2026
9 checks passed
This was referenced May 4, 2026
Merged
Merged
@zihuaihuai
zihuaihuai deleted the fix/ci-and-bugs branch May 4, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Github Related to the Github repository Python On the Python implementation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@zihuaihuai